Skip to content

Linear reachability, take 2 #658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

frankmcsherry
Copy link
Member

@frankmcsherry frankmcsherry commented Mar 24, 2025

Slowly building up more confidence in changing our internal operator reachability implementation. This is a more careful version of #651, which in particular avoids a shadowing bug that seemed to be at the root of #654.

The external facing changes are that customer operator input/output creation no longer take a Vec<Antichain<_>>, but rather an I: IntoIterator<Item=(usize, Antichain<_>)>. This allows a sparse description of connectivity, in cases where it would be expensive to produce a large number of Antichain::default() empty antichains.

The new behavior is scale-tested in tests/shape_scaling.rs, where we test operators and subgraphs up to 100,000 x 100,000.

@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch from 7d8c5c4 to 4929372 Compare March 24, 2025 21:44
@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch 2 times, most recently from fcbac30 to b4ef9b2 Compare March 25, 2025 15:36
@frankmcsherry frankmcsherry force-pushed the linear_reachability_2 branch from b4ef9b2 to f8a234a Compare March 25, 2025 15:49
@frankmcsherry frankmcsherry marked this pull request as ready for review March 26, 2025 02:27
Copy link
Member

@antiguru antiguru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good; left one inline comment.

Co-authored-by: Moritz Hoffmann <antiguru@gmail.com>
@frankmcsherry frankmcsherry merged commit b43381b into TimelyDataflow:master Mar 26, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants